Counterfactual Reasoning and Probabilistic Methods for Trustworthy AI
Go/No Go Meeting 2022
The Problem with Today’s AI
From human to data-driven decision-making …
- Black-box models like deep neural networks are being deployed virtually everywhere.
- Includes safety-critical and public domains: health care, autonomous driving, finance, …
- More likely than not that your loan or employment application is handled by an algorithm.
… where black boxes are recipe for disaster.
- We have no idea what exactly we’re cooking up …
- Have you received an automated rejection email? Why didn’t you “mEet tHe sHoRtLisTiNg cRiTeRia”? 🙃
- … but we do know that some of it is junk.
Explaining Black-Box Models through Counterfactuals
CounterfactualExplanations.jl is a package for generating Counterfactual Explanations (CE) and Algorithmic Recourse (AR) for black-box algorithms. Both CE and AR are related tools for explainable artificial intelligence (XAI). While the package is written purely in Julia, it can be used to explain machine learning algorithms developed and trained in other popular programming languages like Python and R. See below for short introduction and other resources or dive straight into the docs.
Effortless Bayesian Deep Learning through Laplace Redux
LaplaceRedux.jl (formerly BayesLaplace.jl) is a small package that can be used for effortless Bayesian Deep Learning and Logistic Regression trough Laplace Approximation. It is inspired by this Python library and its companion paper.
Endogenous Macrodynamics in AR - motivation
TLDR: We find that standard implementation of various SOTA approaches to AR can induce substantial domain and model shifts. We argue that these dynamics indicate that individual recourse generates hidden external costs and provide mitigation strategies.
Description: In this work we investigate what happens if Algorithmic Recourse is actually implemented by a large number of individuals. The chart below illustrates what we mean by Endogenous Macrodynamics in Algorithmic Recourse: (a) we have a simple linear classifier trained for binary classification where samples from the negative class (y=0) are marked in blue and samples of the positive class (y=1) are marked in orange; (b) the implementation of AR for a random subset of individuals leads to a noticable domain shift; (c) as the classifier is retrained we observe a corresponding model shift (Upadhyay, Joshi, and Lakkaraju 2021); (d) as this process is repeated, the decision boundary moves away from the target class.
We argue that these shifts should be considered as an expected external cost of individual recourse and call for a paradigm shift from individual to collective recourse in these types of situations.
LaplaCE: Realistic and Surrogate-Free Counterfactual Explanations
Good VAE
The results in Figure 2 look great!
Bad VAE
But things can also go wrong …
The VAE used to generate the counterfactual in Figure 3 is not expressive enough.
The counterfactual in Figure 4 is also valid … what to do?
TLDR: Using Linearized Laplace Redux we can compute predictive uncertainty estimates for any neural network (Daxberger et al. 2021). By minimizing predictive uncertainty we can create realistic counterfactual explanations (Schut et al. 2021) without the need for surrogate generative models.
Description: We propose LaplaCE: an effortless way to produce realistic Counterfactual Explanations for deep neural networks (DNN) using Laplace Approximation (LA). To address the need for realistic counterfactuals, existing work has primarily relied on separate generative models to learn the data generating process (e.g. Joshi et al. (2019)). While this an effective way to produce plausible and model-agnostic counterfactual explanations, it not only introduces an significant engineering overhead, but also reallocates the task of creating realistic model explanations from the model itself to the surrogate generative model. Recent work has shown that there is no need for any of this when working with probabilistic models that explicitly quantify their own uncertainty. Unfortunately, most models used in practice still do not fulfill that basic requirement, in which case we would like to have a way to quantify predictive uncertainty in a post-hoc fashion. Recent work on Bayesian Deep Learning has shown that LA can be used effectively in this context. By leveraging this finding we show that it is possible to generate realistic counterfactual explanations, without the need to restrict the class of models or rely on a separate generative model.
Potential venues1: FAccT (Dec ’22), AIES (March ’23), NeurIPS (May ’23), SaTML (Sep ’23)
Counterfactual Explanations for Credit Risk Monitoring in Central Banks
Description: This fall I will give a seminar about Counterfactual Explanations and Algorithmic Recourse at the Bank of England. Bank researchers are interested in applying CE and AR to their bank risk prediction models.
Potential venues:
- Blog post applying recent findings to sovereign default risk dataset.
- Contribution to BoE Staff Working Paper.
Counterfactual Explanations for Regression Problems
Description: The literature on Counterfactual Explanations almost exclusively focuses on classification problems. In Finance and Economics, however, the overwhelming majority of problems involve regression. Hence it is perhaps not altogether surprising that practitioners and researchers in these fields are largely unfamiliar with the CE and instead typically rely on surrogate explanations like LIME and SHAP to explain black-box models. Using Spooner et al. (2021) as a potential starting point, I would be interested in exploring how state-of-the-art CE approaches can be applied to regression problems.
Potential venues: -
More Resources 📚
References
